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ABSTRACT: The increasingly integrative use of images with language in 
many different types of texts in electronic and paper media has created an 
urgent need to go beyond logocentric accounts of literacy and literacy 
pedagogy. Correspondingly there is a need to augment the genre, grammar 
and discourse descriptions of verbal text as resources for literacy pedagogy to 
include descriptions of the meaning-making resources of images. Some 
augmentation along these lines has involved the articulation of Hallidayan 
systemic functional descriptions of language, mainly focussed on verbal 
grammar, with the social semiotic descriptions of the meaning-making 
resources of images described in a grammar of visual design proposed by 
Kress and van Leeuwen. However, current research indicates that 
articulating discrete visual and verbal grammars is not sufficient to account 
for meanings made at the intersection of language and image. This paper 
adopts a systemic functional semiotic perspective in outlining a range of 
different types of such meanings in different kinds of texts, suggesting the 
significance of such meanings in comprehending and composing 
contemporary multimodal texts, and the importance of developing an 
appropriate metalanguage to enable explicit discussion of these meaning- 
making resources by teachers and students. 
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INTRODUCTION 

It is now widely accepted that literacy and literacy pedagogy can no longer be 
confined to the realm of language alone, and that reconceptualizing literacy and 
literacy education needs to account for the role of images (as well as other modes of 
meaning-making) in paper (hard copy) and electronic media texts. In Australia, State 
English syllabi generally require students to learn about the role of images in their 
comprehension, and to a lesser extent, their composition of various kinds of texts. 
This appears to be largely uncontentious in contemporary English teaching. What is, 
and has long been contentious in dealing with language in English teaching in 
Australia, the United Kingdom and North America is the role of metalanguage - the 
type of grammar, its purpose in the curriculum and approaches to its teaching. Today, 
in the national curriculum for England and in English syllabi in Australian States, 
grammar is required to be taught. For the most part, traditional grammar terminology 
has been retained, although some Australian States also incorporate functional 
grammatical concepts from systemic functional linguistics (SEE), sometimes known 
as Hallidayan linguistics (Halliday & Matthiessen, 2004; Martin, 1992). Substantial 
curriculum support documents and appendices to syllabi routinely include quite 
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technical grammatical concepts to facilitate teachers’ and students’ explicit use of this 
metalanguage (Education, 1995; New South Wales Board of Studies, 1998). 

No such comparable accounts of a metalanguage describing the meaning-making 
resources of images and image/text interaction accompany these government 
curriculum documents and syllabi. Faced with the requirement to address the 
multimodality of texts, the prescription of verbal grammar, and the absence in syllabi 
of comparably theorized resources for describing the meaning-making resources of 
images, some teacher educators and teachers have made use of the “grammar of 
visual design” developed by Kress and van Leeuwen (1996), extrapolating from SEE 
accounts of language. The commonality of the systemic functional theoretical 
approach to language and image as social semiotic systems facilitates an articulation 
of visual and verbal grammar as descriptive and analytical resources in developing 
students’ comprehension and composition of multimodal texts. However, beyond 
accounting for the independent, albeit sometimes strategically aligned, contributions 
of language and image to the meaning of composite texts, is the challenge of 
systematically theorising and describing resources for the construction of meaning at 
the intersection of language and image. 

The purpose of this paper is to outline recent work addressing this challenge, and in 
so doing to indicate the pedagogic utility of formulating such a metalanguage of 
multimodality for the development of the multiliteracies education needed by students 
to engage with contemporary multimodal texts and texts of electronic multimedia. 
Firstly, 1 shall invite readers to experience an introductory example of one type of 
meaning made at the intersection of language and image in Anthony Browne’s (1994) 
picture book Zoo. In the next section of the paper 1 will outline the key tenets of 
systemic functional semiotic theory that facilitate its use in describing meaning- 
making resources within and across a variety of modes of meaning including 
language, images, music and gesture. The subsequent section, and main body of the 
paper, will outline recent research dealing with the development of descriptions of 
meaning-making resources of image- language interaction. Finally, 1 will suggest - on 
the basis of research reporting the pedagogic efficacy of the metalanguage of SEE, 
some work on the pedagogic use of the grammar of visual design, and the discussion 
in previous sections of the emerging research on descriptions of image-language 
interaction - that teachers, teacher-educators and researchers consider further the 
pedagogic potential of existing and emerging metalanguage drawing on systemic 
functional semiotic approaches to multimodal texts. 


EXPERIENCING MEANING-MAKING AT THE INTERSECTION OF 
LANGUAGE AND IMAGE 

A very clear example of meaning constructed at the intersection of image and 
language is provided in Anthony Browne’s (1994) picturebook. Zoo. For readers who 
are not familiar with this story and are not able to readily locate a copy, it is possible 
to read the relevant excerpt via the story sample provided on Amazon.com or simply 
go to Amazon.com and search for Zoo. In this story. Mum and Dad and their pre- 
adolescent sons, the narrator and his brother Harry, go to the zoo. In the book, images 
of the family and other visitors to the zoo are on the left-hand side of the double page 
spreads and images of the zoo animals are on the right hand side. In the story 
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segment diseussed here, the image on the left hand side shows a very low angle, 
medium elose view of Dad from the waist up with two white elouds in the sky 
positioned to suggest they are horns protruding from eaeh side of Dad’s head. In the 
text below the image, Harry, the narrator, asks if they ean eat the ehoeolate that Mum 
paeked. Dad refuses, and when asked why, simply says, “Beeause I say so.” The 
image on the right hand side shows the giraffes with no text. On the subsequent left 
hand page we see a rear distanee view of Dad and the boys leaning over a fenee. The 
text below eoneems the tiger they are looking at and makes no mention of the 
ehoeolate or eating. However, in the image on the ground at Dad’s feet it is possible 
to diseem what looks like a disearded ehoeolate wrapper. After reading these pages 
the reader is in a position to suggest why Dad did not allow the boys to have the 
ehoeolate, but to do so s/he must make the inferenee on the basis of eonverging 
information from the image (showing the disearded ehoeolate wrapper) and the text 
that oeeurred two pages earlier. 


KEY TENETS OF SYSTEMIC FUNCTIONAL SOCIAL SEMIOTIC THEORY 

Aeeording to SFL, the struetures of language have evolved (and continue to evolve) 
as a result of the meaning-making functions they serve within the social systems or 
cultures in which they are used. Language is considered as a meaning-making system 
where the options available to individuals to achieve their communicative goals are 
influenced by the nature of the social context and how individuals are positioned in 
relation to it. However, although Halliday focused on language, he was very clear 
that this was only one semiotic system among many other modes of meaning in any 
culture, which might include 

... both art forms such as painting, sculpture, music, the dance, and so forth, and 
other modes of cultural behaviour that are not classified under the heading of forms 
of art, such as modes of exchange, modes of dress, stmctures of the family, and so 
forth. These are all bearers of meaning in the culture. Indeed we can define a culture 
as a set of semiotic systems, as a set of systems of meaning, all of which interrelate 
(Halliday & Hasan, 1985, p. 4). 

The strength of SFL in contributing to frameworks for the development of 
intersemiotic theory emanates from its conceptualization of language as one of many 
different interrelated semiotic systems, and hence the assumption that the forms of all 
semiotic systems are related to the meaning-making functions they serve within social 
contexts. SFL proposes that these meaning-making functions can be grouped into 
three main categories, or metafunctions. These are the three types of meaning- 
making that are inherent in all instances of communication, regardless of whether the 
communication is via language, image, music, sculpture or some other semiotic mode. 
The three kinds of meaning-making or metafunctions are related to three 
corresponding situational variables that operate in all communicative contexts. 

Any communicative context can be described in terms of these three main variables 
that are important in influencing the semiotic choices that are made. The first of these. 
Field, is concerned with the social activity, its content or topic; the second. Tenor, is 
the nature of the relationships among the people involved in the communication; and 
the third. Mode, is the medium and channel of communication. In relation to 
language. Mode is concerned with the role of language in the situation - whether 


English Teaching: Practice and Critique 


57 



L. Unsworth 


Towards a metalanguage for multiliteracies education ... 


spoken or written, aeeompanying or eonstitutive of the aetivity, and the ways in whieh 
relative information value is eonveyed. These situational variables are related to three 
overarehing areas of meaning, or metafunetions: “ideational”, “interpersonal” and 
“textual”. For example, if I say, “My daughter is eoming home this weekend”, 
ideationally this involves an event, a partieipant and the eireumstanees of time and 
plaee assoeiated with it. Interpersonally it eonstruets me as a giver of information and 
the reader/listener as a reeeiver (as well as perhaps suggesting I have at least some 
aequaintanee with the listener). Textually, it loeates “my daughter” as the “Theme” 
or orientation or point of departure for the interaetion, simultaneously suggesting that 
“my daughter” is given information that we both know about (“Given”) and the new 
information is that she is eoming home “this weekend” (“New”). If I say, “Is my 
daughter eoming home this weekend?” the ideational meanings remain the same - the 
event, the partieipant, the eireumstanees have not ehanged. But the interpersonal 
meanings have eertainly ehanged. Now I am demanding information, not giving it 
(and there may be some suggestion of estrangement between the listener and me). 
Similarly, if I say, “This weekend my daughter is eoming home”, the ideational 
meanings are still the same, but this time the textual meanings have ehanged. Now 
the orientation (Theme) is the weekend and this is the given or shared information. 
What is new or unknown eoneems what my daughter is doing. So the different 
struetures refleet different kinds of meaning, whieh in turn refleet different aspeets of 
the eontext. The metalanguage of systemie funetional grammar derives from this 
linking of language strueture, meaning and eontext. 

It is this metafunetional aspeet of SFL and its link to the situational variables of soeial 
eontexts that has provided a eommon theoretieal basis for the development of similar 
“grammatieal” deseriptions of the meaning-making resourees of other semiotie 
modes. For example, Kress and van Leeuwen (1996) proposed that images, like 
language, also always simultaneously realize three different kinds of meanings. 
Images eonstruet not only representations of material reality but also the interpersonal 
interaetion of social reality (sueh as relations between viewers and what is viewed). 
In addition images eohere into textual eompositions in different ways and so realize 
semiotic reality. More teehnieally, the “grammar of visual design” formulated by 
Kress and van Leeuwen (1996) adopted from SFL the metafunetional organization of 
meaning-making resourees: 

• representational/ideational struetures verbally and visually eonstruet the 
nature of events, the objeets and partieipants involved, and the eireumstanees 
in whieh they oeeur. 

• interactive/interpersonal verbal and visual resourees eonstruet the nature of 
relationships among speakers/listeners, writers/readers, and viewers and what 
is viewed. 

• Compositional/textual meanings are eoneerned with the distribution of the 
information value or relative emphasis among elements of the text and image. 

Many researehers exploring image/text relations explieitly aeknowledge the 
grounding of their work in the SFL metafunetional hypothesis (Baldry, 2000; Lemke, 
1998a, 1998b, 2002; Maeken-Horarik, 2003a, 2004; Martin, 2002; O'Halloran, 2004; 
Royee, 1998, 2002). Similar extrapolations from the metafunetional basis of SFL 
have provided soeial semiotie deseriptions of “displayed art” (O'Toole, 1994), musie 
and sound (van Leeuwen, 1999) and aetion (Martinee, 1999, 2000a, 2000b). 
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TOWARDS A METALANGUAGE OF IMAGE-LANGUAGE INTERACTION 

A metafunctional orientation to describing inter-modal meaning 

The formulation of a metalanguage for multiliteraeies needs to entail both the 
deseription of the speeifie eharaeteristies of eaeh partieipatory semiotie mode and also 
the more broadly eneompassing semiotie eharaeteristies that enable it to be related to 
the meaning-making eontributions of other modes in multimodal texts (Kress, 2000a, 
2003a; Maeken-Horarik, 2003a; Martin, 2003). In working towards this formulation 
Kress (2000a; 2003a) eautions against too mueh relianee on deseriptions deriving 
from language-based theories of eommunieation and meaning. 

Aeknowledging the eoneem by Kress that over-relianee on language-based theories of 
meaning and eommunieation would obviate an adequate and integrated deseription of 
multimodal textual objeets, it would nevertheless seem that he and other SFL- 
influeneed soeial semiotieians have effeetively established a mapping of the SFL 
metafunetions aeross modalities. With slight differenees in nomenelature, the 
equivalent of Halliday’s metafunetions have been readily applied to soeial semiotie 
aeeounts of images as summarized in Table 1 adapted from Martin (2002). 


metafunction: 

modalities: 

naturalizing reality 

enacting social relations 

organizing 

text 

verbiage 




Halliday (1994) 

ideational 

interpersonal 

textual 


image 




Kress & van 

Leeuwen (1996) 

representation 

interaction/modality 

composition 

O’Toole (1994) 

representational 

modal 

compositional 

Lemke (1998b) 

presentational 

orientational 

organizational 


Table I. Metafunetions in verbiage and image (after Martin, 2002, p, I) 

Martin (2002) further pointed out that these same modes of meaning had, to some 
extent, been deployed for analyzing relations aeross modalities in multimodal texts. 
For example, he eited the work of Kress and van Leeuwen (1996) on 
textual/eompositional meaning, whieh both adopted and adapted the SFL notion of 
“information foeus”. One aspeet of this is the distinetion between “Given” and 
“New” information. In language, typieally information that is already known or 
familiar to the reader, or “Given”, is loeated at the beginning of the elause (mapped 
onto the Theme), while information that is “New” is loeated at the end of the elause 
(mapped onto the Rheme) (Halliday, 1994). 

The visual analogue of this proposed by Kress and van Leeuwen (1996) is that 
typieally in images and image/text eompositions, for those in Western eultures, the 
Given information is loeated on the left and New information is loeated on the right. 
But Kress and van Leewen (1996) provided further deseriptions of additional 
parameters of information foeus distinetive to images and image/text eompositions. 
For example, they distinguished the top half of sueh eompositions as typieally the 
loeation of the “Ideal” while the lower half was the loeation of the “Real”. In 
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advertisements, the top part typieally indieates the promise of the produet - its 
imagined or ideal effeets, while the bottom part of the layout indieates more eonerete 
information about the produet itself In textbooks, the top part deals with the more 
generalized, abstraet, eoneeptual information, while the bottom part deals with the 
speeifie, eonerete, observable information. It may be, then, that Given or New are 
realized by an image in some instanees and language in others. Similarly, in some 
texts the Ideal is realized by an image and the Real by language or viee versa. 

The ehallenging task of formulating a metalanguage of multiliteraeies ean be very 
usefully informed by the initiatives of soeial semiotie researehers in this direetion 
involving adoption and adaptation of, and innovation on, generative semiotie theory 
with an SFL lineage. The subsequent sub-seetions review reeent examples of sueh 
work dealing in turn with researeh foeusing on eaeh of the three metafunetions. 

Describing resources for the inter-modal construction of ideational meaning 

What is being investigated here is the spaee of integration between language and 
image as soeial semiotie systems in order to provide a theoretieal deseription of the 
dynamies of interaetion between language and image in meaning-making (Lim, 
2004). In terms of ideational meaning, this interaetion may be eharaeterized as 
ideational concurrence (Gill, 2002), complementarity or connection. 

Ideational Concurrence 

Ideational eoneurrenee was deseribed by Gill (2002) in a study of image/text relations 
in pieture storybooks for young ehildren. Coneurrenee referred to ideational 
equivalenee between image and text. This was operationalized as the image and text 
having an equivalent partieipant-proeess-phenomenon eonfiguration. For example, 
the first image in Anthony Browne’s well-known pieturebook. Gorilla (Browne, 
1983), ean be transeoded as. “Hannah is reading a book about gorillas while sitting on 
the floor.” This eoneurs with the verbal text: “She read books about gorillas.”. 
Coneurrenee may entail some form of redundaney aeross modes, but this is not a 
simple inter-modal duplieation of meaning. Although Martinee and Salway (2005) do 
not use the eategory concurrence, they deseribe one sueh type of image-text relations 
as “exposition” - “where the image and the text are of the same level of 
generality”(Martinee & Salway, 2005, p. 350). Their example is the relation between 
an image and its eaption “Light mierograph of a bone” (Martinee & Salway, 2005, p. 
362). 

In examples from ehildren’s literature like the one from Gorilla (Browne, 1983), the 
image-text relation is one of instantiation. The language eonveys the habitual nature 
of the aetivity while the image indieates one instanee, adding to the meaning of the 
language version that, at least on some oeeasions, this reading was done while sitting 
on the floor in the house. The degree of redundaney is variable depending on the 
context of the process or activity common to the language and image. For example, 
image three in Gorilla depicts the father walking along the street with a briefcase. 
This concurs with, and provides an instantiation of the text: “He went to work every 
day.” The image clearly suggests additional meanings such as what kind of work he 
did and to some extent how he got to work. A similar category from the Martinee and 
Salway (2005) work on news websites, textbooks and advertisements is that of 
“exemplification”. This relation obtains when either the image or the text is more 
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general. The former is illustrated by a skull and crossbones image with the caption 
“Kills by biting prey with jagged teeth”. The latter is illustrated by an image of 
children playing and rolling down a hill in a cardboard box accompanied by the 
caption “Remember when total freedom came in a box”. These could also be 
described in terms of instantiation. The children playing in the box is an instance of 
the generality of the caption and the death symbol of the skull and crossbones is 
instantiated by the accompanying text. In both cases, like the children’s literature 
examples, significant additional meanings are added by the language or the image. 

A further means by which ideational concurrence is achieved inter-modally is perhaps 
the most immediately arresting to the reader/viewer. This is the phenomenon of 
“homospatiality”, discussed by Lim (2004), and refers to texts where two different 
semiotic modes co-occur in one spatially bonded homogenous entity. One example 
shows the linguistic representation, “snaaap”, which visually appears with the “sna” 
segment forming one arm of an inverted “v”shape and the “aap” segment forming the 
other arm, so that it appears that the word itself has “snapped”, as indicated in Figure 
1 . 



Figure 1, “homospatiality” 

Another example shows an image of a campfire with the heat arising from the fire 
represented by curved lines, which can be read to spell the word “hot”. 

Ideational concurrence then, is consistent with Lemke’s (2002) notion of the 
multiplicative nature of the meaning-making capacity of multimodal texts being the 
logical product of the capacities of the constituent semiotic systems. In other words 
the visual-verbal interface is synergistic, producing a total effect that is greater than 
the sum of the contributions of each modality (Royce, 1998). At this point we could 
summarize our partial framework for understanding the construction of ideational 
meaning at the intersection of language and image as a set of semantic options for 
intermodal relations as indicated in Figure 2. 
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r redundancy 


concurrence 


exposition 
instantiation — 
homospatiality 


image instantiates text 
text instantiates image 


Figure 2, Ideational Concurrence 


Ideational Complementarity 

Ideational complementarity refers to the situation in multimodal texts where what is 
represented in images and what is represented in language may be different but 
eomplementary and joint eontributors to an overall meaning that is more than the 
meanings eonveyed by the separate modes. Quite independently, Kress in the UK 
(Kress, 1997, 2000b, 2003a, 2003b) and Lemke in the US (Lemke, 1998b) have 
explicated what is referred to as the “functional specialization” of language and 
image. According to this specialization principle, the resources of language are most 
apposite to the representation of sequential relations and the making of categorical 
distinctions, while the resources of images are most apposite to the representation of 
spatial relations and for formulating relationships such as those of degree, gradation, 
continuous co-variation and dynamic emergence (Lemke, 1998b). Language and 
images are not restricted to the areas of representation indicated by the functional 
specialization principle, but as images are becoming more frequent in a wide range of 
texts, functional specialization is likely to characterize the ideational complementarity 
of these two modes. 


One type of ideational complementarity is augmentation - where each of the modes 
provides meanings additional to and consistent with those provided in the other mode. 
Martinec and Salway (2005) refer to this as “extension”, but provide only examples 
that indicate the text adding to the meaning of the images. In a study comparing 
school science explanations in books, on CD ROMs and on the World Wide Web 
(Unsworth, 2004) in terms of the relationships between illustrations and the main text, 
data for the image-text relation of extension included instances where the image 
extended the meanings of the text. For example, the explanation of the water cycle on 
the Classroom of the Future website included evaporation from the soil and the 
movement of clouds in its diagram but did not mention these in the main text. See 
also Unsworth (in press)for instance of images in advertisements extending the 
meanings of the text, such as the advertisement for Mercedes-Benz E-class sports 
pack cars where it is only in the image that we are informed that this vehicle is 
available in sedan and wagon models. 


The augmentation of the text by images is fundamental to the construction of 
interpretive possibilities in literary picture-books for children. This can be seen in 
examples of such picture-books where significant segments of the narrative are 
conveyed by several pages that consist of images alone. In Where the Wild Things Are 
(Sendak, 1962), for example, the conduct of the “wild rumpus” is conveyed by 
images alone in three consecutive double page spreads. But also, where images and 
text are co-present, significant elements of the action of the story frequently occur 
within the images only. For example, on page nine of Anthony Browne’s Gorilla 
(Browne, 1983), the text foreshadows subsequent events: “In the night something 
amazing happened.” Then the images on this page are exclusively responsible for 
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conveying just what the amazing event was. It is these images only that depiet 
Hannah’s toy gorilla growing into a real gorilla (Gill, 2002). Juxtaposed images and 
text in pieture-books have also been shown to jointly eonstruet aetivity sequenees. 
Gill, extending her work on ideational eoneurrenee in pieture-books, deseribed the 
nature of this joint image/text eonstruetion of meaning as “distribution”. Distribution, 
however, seems to be appropriately diseussed within ideational eomplementarity. 
Aeeording to Gill (2002), there are two types of distribution. Intra-proeess 
distribution refers to the portrayal by images and text of different aspeets of a shared 
proeess. For example, the image(s) might depiet the end result of a proeess deseribed 
in the verbal text. This oeeurs in Gorilla when the text indieates that Hannah and the 
gorilla erept downstairs and Hannah put on her eoat and the gorilla put on her father’s 
hat and eoat. The image shows them standing in the doorway so dressed. Inter- 
proeess distribution oeeurs when images fdl a gap in the ideational flow of meaning 
in the verbal text. For example, later in the story of Gorilla, the text indieates that it is 
time to go home and then indieates that they daneed on the lawn, whieh is elearly in 
front of Hannah’s home. But the text makes no referenee to their aetually going 
home. This is eonveyed by the image of the gorilla walking along the street with 
Hannah on his shoulders. 

Another form of Ideational Complementarity is Ideational Divergence, where the 
ideational eontent of text and image are opposed. Ideational divergenee does not 
seem to have figured in the researeh dealing with inter-semiotie eoneurrenee and 
eomplementarity, and it is not mentioned in the system for image-text relations 
proposed by Martenie and Salway (2005). Nevertheless, it is elearly important in 
ehildren’s literary pieture books. For example, in the “Shirley” books by John 
Bumingham (1977; Burningham, 1978), the text and images of Shirley’s parents 
eonvey a narrative of a typieal beaeh visit or of a ehild taking a bath, while the images 
of Shirley depiet her as partieipating in exeiting adventures sueh as her eneounter with 
pirates. Similarly, MeCloud (1994) has drawn attention to the role of ideational 
divergenee in the narrative art of eomie books. In his eategory of image/text relations, 
he uses the term “parallel eombinations” to denote instanees where “words and 
pietures seem to follow very different eourses - without interseeting”(MoCloud, 
1994, p. 154). 

A simple framework summarizing these types of image-text ideational 
eomplementarity is shown in Figure 3. 


complementarity 


P augmentation 
^divergence 


image extends text 
text extends image 


Figure 3. Ideational Complementarity 


Connection 

There are two types of connection between images and text. The first of these is 
known as projection and most eommonly involves the quoting or reporting of speeeh 
or thoughts. The seeond type of eonneetion involves the conjunctive relations of time. 
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place and cause. Martinec and Salway (2005) treat this second type as a further 
category (enhancement) in their description (using different terminology) of ideational 
concurrence and complementarity, and distinguish this whole group of three 
categories (which they call expansion) from projection. This is consistent with SFL 
descriptions of logical relations. My departure from SFL here is exploratory only, in 
the light of the advice from Kress noted earlier and in seeking a felicitous account of 
image-text inter-modal meaning-making. 

Projection in the Martinec and Salway (2005) system refers to either a “locution”, 
which is the quoting or reporting of wording, or an idea, which is the quoting or 
reporting of thought. They cite the speech or thought bubbles in cartoons as the 
typical realizations. But a further realization of projection in language/image 
interaction occurs where a verb in the text “projects” or quotes what a character says 
or is thinking and the verbal or mental quotation is realised by images rather than 
language. It also refers to the juxtaposition of quoted speech and a participant in the 
image represented as the obvious source of the quote. The latter form of projection 
commonly occurs in magazine advertisements where the participant looks directly at 
the viewer from a social or close-up position, thereby making contact that “demands” 
a pseudo interpersonal interaction. The juxtaposed quote then is very strongly 
assumed to be attributed to this represented participant. One example of such an 
advertisement is provided by Cheong (2004). It shows a “demand” image of a smiling 
young woman at a medium close-up position holding a poster with the logo of the 
“Ml” company that offers an attractive, weekly, “off peak” discount for energy 
consumption. The quote spans the width of the advertisement and is located just 
above the head of the woman: “I get the feeling that Ml wants me to enjoy value - 
and enjoy life. Everything they offer is brighter, nicer and more fun!” 

The Economist magazine advertisement analysed by Royce (1998) shows a 
monochrome photograph with a medium to close-up, eye-level view of a young 
woman whose gaze is directed at the viewer, and whose frontal plane is parallel with 
that of the viewer. These visual features realize a pseudo interpersonal relation of 
direct involvement at a personal level with a demand for a response. Positioned 
immediately above this image is the following question in the largest font on the page: 
“Does your environmental policy meet your granddaughter’s expectations?” The 
implicitness of the attribution of the quote combined with the interactive role of the 
choice of image demonstrates the powerful engagement of projection achieved 
through the intersection of language and image. 

Image projection from a verb in the text occurs in the picture book Hyram and B 
(Caswell & Ottley, 2003). In this story Hyram and B are two bears who have lived 
on the shelf in a second-hand shop longer than any other toys. They have shared 
memories of their traumatic days of being discarded and their understandings of 
loneliness. Eventually a young war orphan named Catherine takes B home, turning 
the world of the two bears upside down. But later, Catherine returns to the shop and 
collects Hyram. Two, consecutive, double-page spreads deal with Hyram’ s 
recounting his earlier life to B. In the first of the double page spreads, B says: 
“Hyram sleeps a lot. He told me about it once.” The verb “told” projects what is 
realized both verbally and visually in the next double page spread as Hyram recounts 
his experience; and this past experience is also recalled visually in the illustrations. 
On a later, double-page spread the verb “remember” on the right hand page projects 
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B’s memories represented only by the images on the left hand side of that double 
page and also on the subsequent double page spread. 

A further realization of projeetion is proposed by Martinee and Salway (2005) to 
aeeount for oeeurrenees of a diagram whieh reeapitulates the ideational eontent of a 
juxtaposed segment of main text. However, they do not make it elear whether they 
regard this as a loeution or an idea. Here, projection will be deseribed as verbal when 
the quoting or reporting is eoneemed with wording, and as mental when the eoneem 
is with thought. 

Conjunction refers to the connection of images and text in terms of eausal, temporal 
or spatial relations. The third-last, double-page spread in Hyram and B (Caswell & 
Ottley, 2003) eonstruets eausality at the interseetion of image and text. The left-hand 
third of this double spread shows the following text (in eolumn format) with a rear 
view image of Catherine holding B in her arms with “aetion” lines suggesting that she 
is trembling or that she is roeking B. 

Catherine loves me. 

Catherine understands the secret language of bears. 

She understands what it means to be lonely 

(Caswell & Ottley, 2003, no page numbers). 

The remaining two thirds of the left page and the entire right hand page show an 
explosive warfare seene with fire and a helieopter in the baekground, artillery and a 
damaged tank in the foreground as well as a the rear-view image of a red-headed girl 
in an almost parallel pose to the separate image of Catherine on the far left page onto 
whieh this warfare image is partially superimposed. This parallelism suggests why 
“Catherine understands the seeret language of bears” and why she “understands what 
it means to be lonely”. 

Causal eonjunetion is illustrated by Martinee and Salway (2005) by means of an 
image showing what appears to be people walking around a line up of “body bags” or 
“eoffins aoeompanied by the eaption: “Poliee believe a short eireuit set fire to the 
hall’s thateh roof” The authors elaim here that “the image enhanees the text. The 
dead bodies lying on the floor are the result of a short circuit set fire to the hall’s 
thatch roof' (Martinee & Salway, 2005, p. 351). However, one might also reason 
that the text enhanees the image sinee the short eireuit. . . was the eause of the line-up 
of dead bodies. 

Temporal relations between images and text ean also be seen in pieture-books where 
juxtaposed images and text jointly eonstruet aetivity sequenees. Gill (2002) deseribed 
the nature of this joint image/text eonstruetion of meaning as “distribution”. 
Aeeording to Gill (2002), there are two types of distribution. Intra-proeess 
distribution refers to the portrayal by images and text of different aspeets of a shared 
proeess. For example the image(s) might depiet the end result of a proeess deseribed 
in the verbal text. This oeeurs in the pieture book Gorilla (Browne, 1983), when the 
text indieates that Harmah and the gorilla erept downstairs and Harmah put on her eoat 
and the gorilla put on her father’s hat and eoat. The image shows them standing in 
the doorway so dressed. Inter-proeess distribution oeeurs when images fdl a gap in 
the ideational flow of meaning in the verbal text. For example, later in the story of 
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Gorilla, the text indieates that it is time to go home and then indieates that they 
daneed on the lawn, whieh is elearly in front of Hannah’s home. But the text makes 
no referenee to their aetually going home. This is eonveyed by the juxtaposed image 
of the gorilla walking along the street with Hannah on his shoulders. 

Martinee and Salway (2005) illustrate temporal relations between image and text with 
a segment showing an image of one of Max Beekmann’s paintings and an 
aeeompanying main text whieh begins: 

Beckmann worked for the German army’s medical corps during the war, sketching 
the horrors of what he saw. Following a nervous breakdown, his paintings became 
harsher... (Martinee & Salway, 2005, p. 351). 

The authors indieate that “Following a nervous breakdown’’ situates in time the 
example of Beekmann’s paintings provided by the image. Martinee and Salway 
(2005) illustrate enhaneement by plaee with an image of Neweastle Airport (with the 
name and loeation of the airport on a sign in the image) and the following 
aeeompanying eaption: The woman arrived too late to board the flight to Paris 
(Martinee & Salway, 2005, p. 350). 

The framework deseribing meaning made at the interseetion of image and language 
through connection ean be summarized in Figure 4. 


connection 


- projection 
^ conjunction 


verbal 

mental 

causal 

temporal 

spatial 


Figure 4. Connection 

An overall framework deseribing ideational meaning-making at the interseetion of 
image and language ean be summarized as indieated in Figure 5. 

Image-text relations in the construction of interpersonal meaning 

Interactive and Evaluative Meaning 

Interpersonal meaning in SFL includes interactive and evaluative meaning. 
Interactive meaning refers to the roles of interactants in giving information (making 
statements) or providing goods and services (making offers) or demanding 
information (asking questions) or ordering goods and services (giving commands). 
These interactive roles are realized grammatically by the mood system (Halliday, 
1994). The grammar of visual design proposed by Kress and van Leeuwen (1996) 
indicates that visually only two interactive roles can be portrayed: a “demand’’ image 
has the gaze of one or more represented participants directed to the viewer and hence 
“demands” some kind of response in terms of the viewer entering into some kind of 
pseudo-interactive relation with the represented participant; an “offer” does not have 
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the gaze of any represented participant directed to the viewer and hence provides a 
portrayal for the viewer’s contemplation. 


rconcurrence 


redundancy 

- exposition 

- instantiation 

- homospatiality 


image instantiates text 
_ text instantiates image 


Ideational 
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and image 
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image extends text 
- text extends image 


Lconnection 


projection 


L conjunction 


^ verbal 
mental 
causal 

temporal 

-Spatial 


Figure 5, Ideational meanings at the intersection of language and image 

Evaluative meaning in SEE has traditionally been confined to commentary on the 
truth of what is represented linguistically. This is realized by polarity (yes or no) and 
by the system of modality, which realizes possibilities between positive and negative 
polarity, such as degrees of certainty and probability (perhaps/of course; 
possibly/probably/certainly), and degrees of usuality and frequency (sometimes/ 
usually/always). In the grammar of visual design, evaluation also focuses on the truth 
or credibility of images, also referred to as modality (Kress & van Leeuwen, 1996). 
Modality value, however, is related to “coding orientation’’. Within a naturalistic 
coding orientation, high modality is a reflection of the fidelity of the representation 
with the natural world, such as that achieved in a high-quality, colour photograph. 
Within a scientific coding orientation, fidelity may be calibrated more in relation to 
the representation of conceptual clarity rather than naturalistic reality. 

Martin has extended SEE perspectives on evaluation by proposing an “appraisal 
network’’ including three main systems - attitude, engagement and graduation 
(Martin, 2000; Martin & Rose, 2003). This work has also been made available in a 
form easily accessible to teachers and students (Droga & Humphrey, 2002). Here I 
will deal with the category of attitude only. Within attitude there are a number of sub- 
categories: Affect refers to the expression of feelings, which can be positive or 
negative, and may be descriptions of emotional states (for example, happy) or 
behaviours that indicate an emotional state (for example, “crying”). Sub-categories 
of Affect are “happiness”, “security” and “satisfaction”. Appreciation relates to 
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evaluations of objects, events or states of affairs and can also relate to the 
characteristics of people but not their behaviour. Appreciation is further subdivided 
into “reaction”, “composition” and “valuation”. Reaction involves the emotional 
impact of the phenomenon (for example, thrilling, boring, enchanting, depressing). 
Composition refers to the form of an object (for example, coherent, balanced, 
haphazard) and valuation refers to the significance of the phenomenon (for example, 
groundbreaking, inconsequential). Judgment can refer to assessments of someone’s 
capacities (brilliant, slow), their dependability (tireless, courageous, rash) or their 
relative normality (regular, weird). Judgment can also refer to someone’s truthfulness 
(frank, manipulative) and ethics (just, cruel, corrupt). Recent research on 
interpersonal meaning in image/text relations has noted the joint construction of 
interaction, but the main impact of these inter-modal relations from an interpersonal 
perspective seems to be oriented to the construction of evaluative stance in 
multimodal texts. 

Portraying interpersonal interaction through image-text relations 

The Economist magazine advertisement analysed by Royce (1998) shows a 
monochrome photograph with a medium to close-up, eye-level view of a young 
woman whose gaze is directed at the viewer, and whose frontal plane is parallel with 
that of the viewer. These visual features realize a pseudo interpersonal relation of 
direct involvement at a personal level with a demand for a response. Positioned 
immediately above this image is the following question in the largest font on the 
page: “Does your environmental policy meet your granddaughter’s expectations?” 
Royce points out the ways in which this question, with its second person address and 
similar features in the subsequent text, effects a joint image/text initiation of 
interaction, which he refers to as “Reinforcement of Address”. Similar work by 
Cheong (2004) shows how the medium to close-up, eye-level demand image of a 
smiling young woman whose frontal plane parallels that of the viewer is juxtaposed 
with the written text positively evaluating the products of the Ml telecommunications 
company, so that she appears to be the speaker of the quotation: “I get the feeling that 
Ml wants me to enjoy value - and enjoy life. Everything they offer is brighter, nicer 
and more fun!” In texts of this kind, the image/text relations are jointly constructing 
evaluative stance as well as interaction. 

Communicating evaluative stance through image-text relations 
Gill found that interpersonal alignment could occur across image/verbiage 
juxtapositions, which she described as a resonance of interpersonal meaning. For 
example, on pages 27-28 of Anthony Browne’s (1983), Gorilla, the ideational content 
does not concur. However, there is a resonance between the image and text 
construction of the affect portrayed between Hannah and her father. In the text the 
father says: “Happy birthday, love”, and the image shows Ha nn ah with her father 
putting his hands on Hannah’s shoulders. Gill’s analysis showed many examples of 
resonance of appraisal content, such as Affect. For example, on page eleven of 
Gorilla, the text indicates: “Hannah was frightened”, corresponding to the image of a 
frightened Hannah with the bedclothes drawn up over part of her face. On pages 17- 
18, where Hannah and the gorilla visit the orang-utan and the chimpanzee in the zoo, 
the text indicates: “She thought they were beautiful. But sad,” - corresponding 
visually to the expression of the orang-utan and to a lesser extent that of the 
chimpanzee. Similarly, instances of interpersonal resonance with appraisal content 
were found in the picture book, the baby who wouldn 7 go to bed (Cooper, 1996). For 
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example, the images consistently depict the other participants as looking down on the 
baby from a high vertical angle, positioning them as having power over her. This 
concurs with the mood structure of the text, where the other participants make 
statements that serve as indirect, disciplinary comments about the baby’s behaviour. 

As far as interpersonal meaning is concerned, verbiage/image relations in multimodal 
texts, according to Martin (2002), are more concerned with appraisal than with mood 
or modality. He argues that a key function of images is to co-articulate attitude 
(including Affect, Judgment and Appreciation). In doing so, images operate in a 
similar way to imagery, provoking an evaluative reaction in readers, and the images 
are typically positioned to do this so that they preview or foreshadow the value 
positions to be constructed in the subsequent verbiage. One example is taken from 
Nelson Mandela’s The illustrated long walk to freedom (Mandela, 1996). In the 
section dealing with the 1976 Soweto uprising, the well-known photo of the body of 
thirteen-year-old Hector Pieterson being carried from the fray is positioned as a full- 
page image on page 147, preceded by its caption in the right hand margin of the 
previous page. The main text dealing with the Soweto uprising then appears overleaf 
on page 148. The photo previews and amplifies the reaction induced by Mandela’s 
verbal imagery. In SFL terms, Martin suggests that the photo functions as an 
evaluative interpersonal Theme, naturalizing the stance from which the remaining 
verbiage can be read. Additional examples are provided by Martin’s (2002) analyses 
of other sections of this text and further examples from his analyses of the Australian 
Government Report (1997) Bringing them home on the generations of Aboriginal 
children taken from their families and placed in alternative care. This report similarly 
deploys images and imagery to establish evaluative orientations to the ensuring text. 
On the basis of this work, Martin (2002) suggests that for multimodal texts the 
Given/New elements of the compositional meaning-making resources of images, 
extrapolated by Kress and van Leeuwen (1996) from SFL, need to be augmented to 
include a visual version of the SFL concept of Interpersonal Theme. As Martin 
reasons: 

The left is not simply Given, but has a positive forward looking function, instigating 
an naturalizing a reading position for the evaluation of verbiage/image texture that 
ensues (Martin, 2002, p. 334). 

Textual/Compositional Meanings in Image/Text relations 

The descriptions of compositional meanings in images by Kress and van Leeuwen 
(1996) have been extensively applied by them to image/text relations. Further studies 
of school science books have shown how layout resources of Given/New, Ideal/Real, 
and Framing are deployed to structure pedagogic texts (Veel, 1998). Typically, what 
is likely to be familiar to students, whether in the form of language or image, is placed 
in the Given position on the left and that which deals either visually or verbally with 
unfamiliar, technical information is placed in the New position on the right. While 
these Given/New structures are consistent with the usual left to right progression in 
reading, the Ideal/Real structures in school curriculum texts do not necessarily map 
strategically onto our practice in working from top to bottom of the text. Students 
might be advised to examine the specific, concrete information of the Real positioned 
at the bottom of the layout before addressing the more abstract, conceptual, and 
generalized information of the Ideal positioned at the top. Often the salience of 
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concrete images in the Real will influenee students to adopt sueh a reading path 
(Unsworth, 2001). 

It has been noted, in Martin’s (2002) work above, that the deseriptions of the 
compositional meanings in multimodal texts need to be extended to take aeeount of 
the role of images as Interpersonal Theme. Further extensions are suggested in the 
work by Jewitt (2002) dealing with the eompositional resourees for eonstrueting 
charaeter in the Novel as CD ROM version of Steinbeek’s Of mice and men 
(SteinbeekSeries, 1996), and by issues of “framing” raised in Maeken-Horarik’s 
(2003b) study of texts whieh were eentral to the ehildren overboard affair. 

From her Of mice and men study of the Novel as CD ROM, Jewitt (2002) hs 
suggested that the spatial relationship between image and verbiage on eaeh of the 
sereens is itself a meaning-making resouree. She argued that writing serves as a 
visual element, a bloek of “spaee” that makes textual meaning beyond its eontent. 
Jewitt indieated that on the CD sereens, the bloeks of writing were positioned in 
different plaees: the left or right side, along the bottom or top length of the sereen, or 
in the top or bottom eorner. The size and position of the bloek and its loeation 
combined to reveal or eoneeal different parts of the image layered “beneath it”. In 
this way, a bloek of writing emphasizes different aspeets of the image on sereen. 
Aeeording to Jewitt, the image at times euts aeross the lexis and grammar of the 
written text to ereate a visual mood and rhythm, whieh she illustrated with one image 
of George and Lenny that runs aeross four sereens of ehanging text: 

In the first screen, the block of writing sits above George’s head as he talks to Lennie 
about what he could do if he left him. In the second screen Lennie is visually 
obliterated by George's angry talk of leaving, visually foregrounding George. In the 
third screen, as George's anger subsides, the block of writing is placed on the screen 
so that both George and Lennie are visible (Jewitt, 2002, p. 184). 

Jewitt suggested that it is through the visual arrangement of image and writing on 
sereen that the narrative eonstruet of eharaeter indieated intensity of emotion to 
suggest the alignment of the viewer with George’s point of view, and to emphasise 
the ageney/passivity of the eharaeters in the novel. Whether negotiating the meaning 
of newspaper stories, or literary narratives (in book or eleetronie media), layout 
features sueh as framing are erueial elements in the interpretation of the meanings at 
stake and in establishing the evaluative stanee of the writer in relation to those 
meanings (Maeken-Horarik, 2003b; Unsworth, 2006a). 

Although the ideational, interpersonal and eompositional perspeetives on the 
meaning-making resourees of image/language interaetion have been diseussed 
separately here, it must be remembered that in reality these meanings are always made 
simultaneously in all texts, and eritieal understanding of the interpretive possibilities 
of texts needs to be based on an integrative view of all three perspeetives. 
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CONCLUSION: THE PEDAGOGIC POTENTIAL OF A SYSTEMIC 

FUNCTIONAL PERSPECTIVE ON METALANGUAGE AND 
MULTIMODALITY 

There seems to be signifieant support for the view that the need to redefine literaey in 
the eleetronie age entails the development of a metalanguage that will faeilitate 
metatextual awareness of image/text relations (Kamil et al, 2000; Kress, 2003b; 
Maeken-Horarik, 2004; Riehards, 2001; Russell, 2000). Metalanguage entails 
systematie, teehnieal knowledge of the ways in whieh the resourees of language and 
images (and other semiotie systems) are deployed in meaning making. English 
syllabi eurrently require a signifieant eommitment by teaehers and students to 
understanding and using metalanguage. Sueh an investment in teaehing and learning 
ean be produetive if the metalanguage funetions as a tool to enhanee the development 
of eritieal soeial literaeies. 

For this to happen, the metalanguage must be based on systematie aeeounts of the 
meaning-making potential of the multimodal nature of eontemporary texts and also be 
eapable of expansion/modifieation in response to the expansion of meaning-making 
potential with the ongoing emergenee of new forms of eommunieation. This paper 
suggests that systemie funetional semiotie theory has mueh to offer in this respeet. 
However, the work on grammars for exploring the eo-artieulation of image and 
verbiage is in its infaney (Kress, 2001; Maeken-Horarik, 2003a). Little elassroom 
researeh has been done on the pedagogie use of sueh emerging grammars, although 
there is some evidenee that young ehildren ean learn and produetively use aspeets of 
Kress and van Leeuwen’s visual grammar in work with pieture-books and with 
multimedia CD ROMs in eurrieulum area learning (Callow & Zammit, 2002; Howley, 
1996). There is also a good deal of evidenee for the effieaey of the metalanguage of 
SFL in literaey development and learning in primary/elementary and seeondary/high 
sehool eontexts (Quinn, 2004; Sehleppegrell, 2004; Sehleppegrell et al, 2004; Torr & 
Harman, 1997; Williams, 1999, 2000). What is suggested here is that the theoretieal 
bases of the soeial semiotie researeh arising from SFL are providing a generative and 
inelusive framework for the transdiseiplinary development of a metalanguage of 
multiliteraeies. 

While the researeh on an evolving metalanguage of multimodality is in the very early 
stages and emerging deseriptions remain quite tentative, there are at least two, firm, 
praetieal implieations for English teaehers. The first is the robustness, broad 
applieation, and praetieal usefulness of the metafunetional prineiple deriving from 
SFL. That is the prineiple that all texts, visual and verbal, separately and in 
eombination, always simultaneously entail ideational, interpersonal and 
textual/eompositional meanings. This prineiple is frequently refleeted in the rationale 
of the English syllabi of different sehool systems. For example, the eurrent English 1- 
10 English Currieulum for Queensland Sehools in Australia indieates in its rationale: 

We use language purposefully to represent experiences of real and imagined worlds, 

to interact with others, and to create coherent and cohesive texts 

(QueenslandStudies Authority, 2005, p. 1). 
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The metafunctional principle is widely accepted as central to our understanding of 
contemporary and emerging forms of multimodal texts and provides a sound and 
accessible basis for English teachers to further examine the pedagogic potential of 
metalanguage. Detailed accounts of the ways in which SEE and the grammar of 
visual design can be used together in the English classroom are now well documented 
(Callow, 1999; Christie, 2005; Christie & Unsworth, 2005; Cope & Kalantzis, 2000; 
Goodman & Graddol, 1996; Jewitt, 2005; Jewitt & Kress, 2003; Kress, 2003b; 
Unsworth, 2001, 2006b; Unsworth et al, 2005). 

The second practical implication for teachers is their essential role as participants in 
the collaborative researching, theorising and re-formulating of our “metasemiotic” 
understanding of emergent, multimodal text-forms and the concomitant derivation of 
an evolving metalanguage of multimodality to enhance practical pedagogy. The 
interface of teaching and research has long been an essential characteristic of SEE 
work (Christie, 2005; Christie & Unsworth, 2005), and this continues to be the case in 
a great deal of the current research contributing to multimodal text description 
(Macken-Horarik, 1996, 1998, 2003a, 2003b, 2004). It is hoped that this article, by 
stimulating critically constructive responses to, and envisioning beyond, what is 
presented here, will encourage further collaborative work among teachers, teacher 
educators and researchers in other disciplines in exploring the nature and role of a 
metalanguage that will facilitate development of the multiliteracies pedagogies 
appropriate to the multimedia world of our children in the Twenty-First Century. 

Are there any sustainable arguments for a positive relationship between knowledge 
about language (however understood) and increased effectiveness in some aspect of 
textual practice (reading/viewing or production)? 


REFERENCES 

Baldry, A. (Ed.). (2000). Multimodality and multimediality in the distance learning 
age. Campobasso, Italy: Palladino Editore. 

Browne, A. (1983). Gorilla. Eondon: Julia MacRae. 

Browne, A. (1994). Zoo. Eondon: Random House. 

Burningham, J. (1977). Come away from the water, Shirley. Eondon: Cape. 

Bumingham, J. (1978). Time to get out of the bath, Shirley. Eondon: Cape. 

Callow, J. (Ed.). (1999). Image matters: Visual texts in the classroom. Sydney: 
Primary English Teaching Association. 

Callow, J., & Zammit, K. (2002). Visual literacy: From picture books to electronic 
texts. In M. Monteith (Ed.), Teaching primary literacy with ICT (pp. 188-201). 
Buckingham: Open University Press. 

Caswell, B., & Ottley, M. (2003). Hyram and b. Sydney: Hodder Headline. 

Cheong, Y. (2004). The construal of ideational meaning in print advertisements. In K. 
O'Halloran (Ed.), Multimodal discourse analysis: Systemic functional 

perspectives (pp. 163-195). Eondon and New York: Continuum. 

Christie, F. (2005). Language education in the primary years. Sydney: University of 
New South Wales Press. 

Christie, F., & Unsworth, E. (2005). Developing dimensions of an educational 
linguistics. In J. Webster, C. Matthiessen & R. Hasan (Eds.), Continuing 


English Teaching: Practice and Critique 


72 



L. Unsworth 


Towards a metalanguage for multiliteracies education ... 


discourse on language: A functional perspective (Vol. 1, pp. 217-250). London: 
Equinox. 

Commission, H. R. & E. O. (1997). Bringing them home: National inquiry into the 
separation of Aboriginal and Torres Strait Islander children from their families. 
Sydney: Human Rights and Equal Opportunities Commission. 

Cooper, H. (1996). The baby who wouldn't go to bed. London: Doubleday/Pieture 
Corgi Books. 

Cope, B., & Kalantzis, M. (Eds.). (2000). Multiliteracies: Literacy learning and the 
design of social futures. Melbourne: Maemillan. 

Droga, L., & Humphrey, S. (2002). Getting started with functional grammar. 
Marriekville, Australia: Target Texts. 

Edueation, Q. D. of (1995). English 1-10 syllabus: A guide to analysing texts. 
Brisbane: Queensland Government Printing Offiee. 

Gill, T. (2002). Visual and verbal playmates: An exploration of visual and verbal 
modalities in children 's picture books. Unpublished B.A. (Honours), University 
of Sydney. 

Goodman, S., & Graddol, D. (1996). Redesigning English: New texts, new identities. 
Eondon: Routledge. 

Halliday, M. (1994). An introduction to functional grammar (2 ed.). London: Edward 
Arnold. 

Halliday, M., & Hasan, R. (1985). Language, context and text: Aspects of language in 
a social-semiotic perspective. Geelong: Deakin University Press. 

Halliday, M., & Matthiessen, C. (2004). An introduction to functional grammar (3rd 
ed.). London: Arnold. 

Howley, P. (1996). Visual literacy: Semiotic theory, primary school syllabus 
documents and classroom practice. Unpublished Baehelor of Edueation 
Honours thesis. University of Sydney, Sydney. 

Jewitt, C. (2002). The move from page to sereen: The multimodal reshaping of sehool 
English. Visual Communication, 7(2), 171-196. 

Jewitt, C. (2005). Technology, literacy, learning. London: Routledge. 

Jewitt, C., & Kress, G. (Eds.). (2003). Multimodal literacy. New York: Peter Eang. 

Kamil, M., Intrator, S., & Kim, H. (2000). The effeets of other teehnologies on 
literaey and learning. In M. Kamil, P. Mosenthal, P. Pearson & R. Barr (Eds.), 
Handbook of reading research (Vol. 3, pp. 771-788). Mahwah, New Jersey: 
Erlbaum. 

Kress, G. (1997). Visual and verbal modes of representation in eleotronieally 
mediated eommunieation: The potentials of new forms of text. In 1. Snyder 
(Ed.), Page to screen: Taking literacy into the electronic era (pp. 53-79). 
Sydney: Allen and Unwin. 

Kress, G. (2000a). Design and transformation: New theories of meaning. In B. Cope 
& M. Kalantzis (Eds.), Multiliteracies: Learning literacy and the design of 
social futures (pp. 153-161). Melbourne: Maemillan. 

Kress, G. (2000b). Multimodality. In B. Cope & M. Kalantzis (Eds.), Multiliteracies: 
Literacy learning and the design of social futures (pp. 182-202). Melbourne: 
Maemillan. 

Kress, G. (2001). Soeiolinguisties and soeial semioties. In P. Cobley (Ed.), Semiotics 
and linguistics (pp. 66-82). London: Routledge. 

Kress, G. (2003a). Genres and the multimodal produetion of “seientifieness”. In C. 
Jewitt & G. Kress (Eds.), Multimodal literacy (pp. 173-186). New York: Peter 
Eang. 


English Teaching: Practice and Critique 


73 



L. Unsworth 


Towards a metalanguage for multiliteracies education ... 


Kress, G. (2003b). Literacy in the new media age. London: Routledge. 

Kress, G., & van Leeuwen, T. (1996). Reading images: A grammar of visual design. 
London: Routledge. 

Lemke, J. (1998a). Metamedia literaey: Transforming meanings and media. In D. 
Reinking, M. MeKenna, L. Labbo & R. Kieffer (Eds.), Handbook of literacy 
and technology: Transformations in a post-typographic world (pp. 283-302). 
New Jersey: Erlbaum. 

Lemke, J. (1998b). Multiplying meaning: Visual and verbal semioties in seientifie 
text. In J. R. Martin & R. Veel (Eds.), Reading science: Critical and functional 
perspectives on discourses of science (pp. 87-113). Eondon: Routledge. 

Lemke, J. (2002). Travels in hypermodality. Visual Communication, 7(3), 299-325. 

Lim, V. F. (2004). Developing an integrative multi-semiotie model. In K. O'Halloran 
(Ed.), Multimodal discourse analysis: Systemic functional perspectives (pp. 
220-246). Eondon and New York: Continuum. 

Maeken-Horarik, M. (1996). Eiteraey and learning aeross the eurrieulum: Towards a 
model of register for seeondary sehool teaehers. In R. Hasan & G. Williams 
(Eds.), Literacy in society (pp. 232-278). Harlow: Addison Wesley Eongman. 

Maeken-Horarik, M. (1998). Exploring the requirements of eritieal literaey: A view 
from two elassrooms. In F. Christie & R. Misson (Eds.), Literacy and schooling 
(pp. 74-103). Eondon: Routledge. 

Maeken-Horarik, M. (2003a). A telling symbiosis in the diseourse of hatred: 
Multimodal news texts about the “ehildren overboard” affair. Australian Review 
of Applied Linguistics, 26(2), 1-16. 

Maeken-Horarik, M. (2003b). Working the borders in raeist diseourse: The ehallenge 
of the “ehildren overboard affair” in news media texts. Social Semiotics, 13(3), 
283-303. 

Maeken-Horarik, M. (2004). Interaeting with the multimodal text: Refleetions on 
image and verbiage in artexpress. Visual Communication, 5(1), 5-26. 

Mandela, N. (1996). The illustrated long walk to freedom: The autobiography of 
Nelson Mandela. Eondon: Eittle, Brown and Company. 

Martin, J. (1992). English text: System and structure. Amsterdam: Benjamins. 

Martin, J. (2000). Beyond exehange: Appraisal systems in English. In S. Hunston & 
G. Thompson (Eds.), Evaluation in text: Authorial stance and the construction 
of discourse (pp. 142-175). Oxford: Oxford University Press. 

Martin, J. (2002). Fair trade: Negotiating meaning in multimodal texts. In P. Coppoek 
(Ed.), The semiotics of writing: Transdisciplinary perspectives on the 
technology of writing (pp. 311-338). Begijnhof, Belgium: Brepols & Indiana 
University Press. 

Martin, J. (2003). Voieing the “other”: Reading and writing indigenous Australians. 
In G. Weiss & R. Wodak (Eds.), Critical discourse analysis: Theory and 
interdisciplinarity (pp. 199-219). Eondon: Palgrave. 

Martin, J., & Rose, D. (2003). Working with discourse: Meaning beyond the clause 
(1st ed. Vol. 1). Eondon/New York: Continuum. 

Martinee, R. (1999). Cohesion in aetion. Semiotica, 1/2, 161-180. 

Martinee, R. (2000a). Rhythm in multimodal texts. Leonardo, 55(4), 289-297. 

Martinee, R. (2000b). Types of proeess in aetion. Semiotica, 750(3/4), 243-268. 

Martinee, R., & Salway, A. (2005). A system for image-text relations in new (and 
old) media. Visual Communication, 4(3), 337-371. 

MeCloud, S. (1994). Understanding comics: The invisible art. New York: Harper 
Collins. 


English Teaching: Practice and Critique 


lA 



L. Unsworth 


Towards a metalanguage for multiliteracies education ... 


New South Wales Board of Studies. (1998). English K-6 syllabus and support 
documents. Retrieved 7th September, 2005, from 

http://k6.boardofstudies.nsw.edu.au/english/english index.html 

O'Halloran, K. (Ed.). (2004). Multimodal discourse analysis: Systemic functional 
perspectives. London and New York: Continuum. 

O'Toole, M. (1994). The language of displayed art. London: Leicester University 
Press. 

Queensland Studies Authority. (2005). Years 1-10 English syllabus. Retrieved 7th 
September, 2005, from 

http://www.qsa.qld.edu.au/vrs Ito 1 0/kla/english/svllabus.html 

Quinn, M. (2004). Talking with Jess: Looking at how metalanguage assisted 
explanation writing in the middle years. Australian Journal of Language and 
Literacy, 27(3), 245-261. 

Richards, C. (2001). Hypermedia, internet communication, and the challenge of 
redefining literacy in the electronic age. Language Learning and Technology, 
4{1), 59-77. 

Royce, T. (1998). Synergy on the page: Exploring intersemiotic complementarity in 
page-based multimodal text. Japan Association Systemic Functional Linguistics 
Occasional Papers, 7(1), 25-50. 

Royce, T. (2002). Multimodality in the TESOL classroom: Exploring visual-verbal 
synergy. TESOL Quarterly, 36(2), 191-205. 

Russell, G. (2000). Print-based and visual discourses in schools: Implications for 
pedagogy. Discourse: Studies in the Cultural Politics of Education, 27(2), 
205217-. 

Schleppegrell, M. (2004). The language of schooling: A functional linguistic 
perspective. Mahwah, New Jersey and London: Erlbaum. 

Schleppegrell, M., Achugar, M., & Oteiza, T. (2004). The grammar of history: 
Enhancing content-based instruction through a functional focus on language. 
TESOL Quarterly, 35(1), 67-93. 

Sendak, M. (1962). Where the wild things are. London: The Bodley Head. 

SteinbeckSeries. (1996). Of mice and men. New York: Penguin Electronics. 

Torr, J., & Harman, J. (1997). Literacy and the language of science in year one 
classrooms: Implications for children's learning. Australian Journal of 
Language and Literacy, 20(3), 222-231 . 

Unsworth, L. (2001). Teaching multiliteracies across the curriculum: Changing 
contexts of text and image in classroom practice. Buckingham, United 
Kingdom: Open University Press. 

Unsworth, L. (2004). Comparing school science explanations in books and computer- 
based formats: The role of images, image/text relations and hyperlinks. 
International Journal of Instructional Media, 37(3), 283-301. 

Unsworth, L. (2006a). Describing meaning-making at the intersection of language 
and image: Towards a metalanguage for multi-modal literacy pedagogy. Paper 
presented at the Future Directions in Literacy, University of Sydney. 

Unsworth, L. (2006b). E-literature for children: Enhancing digital literacy learning. 
London and New York: Routledge/Falmer. 

Unsworth, L. (in press). Explicating inter-modal meaning-making in media and 
literary texts: Towards a metalanguage of image/language relations. In A. Bum 
& C. Durrant (Eds.), Media teaching: Language, audience, production. London: 
AATE-NATE/Wakefield Press. 


English Teaching: Practice and Critique 


75 


L. Unsworth 


Towards a metalanguage for multiliteracies education ... 


Unsworth, L., Thomas, A., Simpson, A., & Asha, J. (2005). Children's literature and 
computer based teaching. London: McGraw-Hill/Open University Press, 
van Leeuwen, T. (1999). Speech, music, sound. London: Maemillan. 

Veel, R. (1998). The greening of sehool seienee: Eeogenesis in seeondary elassrooms. 
In J. Martin & R. Veel (Eds.), Reading science: Functional and critical 
perspectives on the discourses of science (pp. 114-151). Eondon: Routledge. 
Williams, G. (1999). Children beeoming readers: Reading and literaey. In P. Hunt 
(Ed.), Understanding children's literature (pp. 151-162). Eondon: Routledge. 
Williams, G. (2000). Children's literature, ehildren and uses of language deseription. 
In E. Unsworth (Ed.), Researching language in schools and communities: A 
functional linguistic perspective (pp. 111-129). Eondon: Cassell. 

Manuscript received: Eebruary 16, 2006 
Revision received: May 1, 2006 
Accepted: May 5, 2006 


English Teaching: Practice and Critique 


76 



